1. Technical field of the Invention
The present invention relates to an apparatus which realizes picture coding, for example, moving picture coding defined by a standard such as ISO/IEC 13818-2 (MPEG-2) and, more particularly, to a high picture quality coding control system.
2. Description of the Prior Art
Hitherto, in the case of performing motion compensated interframe predictive coding defined by a standard such as ISO/IEC 13818-2 (MPEG-2), generally, the three kinds of picture types of an intraframe coded picture (I-picture (I frame)), an interframe forward predictive coded picture (P-picture (P frame)), and an interframe bidirectionally predictive-coded picture (B-picture (B frame)) are arranged in an ordered pattern which is generally fixed.
It is common in the technical field to which the present invention belongs that the distance between the I-pictures (I frames) is expressed by xe2x80x9cNxe2x80x9d and the distance between the I-picture and the P- picture (P-frame) or between the P-pictures is expressed by xe2x80x9cMxe2x80x9d. Generally, the N and M values are fixed and are controlled so as not to be dynamically changed.
For example, when the M value is set to 1 as shown in FIG. 9A, the I-picture (I frame) is included as the first picture in a pattern, the P-pictures (P frames) are arranged every (first) picture and no B-picture (B frame) is used. In the case of M=1, a P-picture (P frame) is coded by interframe forward prediction by referring to the immediately preceding picture.
When the M value is set to 2 as shown in FIG. 9B, the I picture is included as the first picture in a pattern, the P pictures are arranged every second picture and one B picture is arranged between each xe2x80x9cI- and P-pictures (P frames)xe2x80x9d and xe2x80x9cP-and P-pictures (P frames)xe2x80x9d. In the case of M=2, each of the P pictures is coded by interframe prediction by referring to the two preceding I- or P-picture (P frame). The B-picture is coded by bidirectional interframe prediction by referring to the forward or backward I- or P-picture (P frame). Consequently, the order of coding by using B-pictures is different from the input order of pictures. After reference frames on both sides are coded, a B-picture is coded.
When the M value is set to 3 as shown in FIG. 9C, the I-picture (I frame) is included as the first picture of a pattern, the P-pictures (P frames) are arranged every third picture, and two B-pictures are arranged between each xe2x80x9cI- and P-pictures (P frames)xe2x80x9d and xe2x80x9cP- and P-pictures (P frames)xe2x80x9d. In the case of M=3, the P-picture (P frame) is coded by interframe prediction by referring to a three preceding I- or P-picture (P frame). The B-picture is coded by bidirectional interframe prediction by referring to the forward or backward I- or P-picture (P frame) in a manner similar to the case of M=2.
Even when the M value increases, the order is similar to the above. The I-picture (I frame) is used as the first picture of a pattern and P-pictures (P frame) are arranged every Mth order. (M-1) B-pictures are used between each xe2x80x9cI- and P-pictures (P frames)xe2x80x9d and xe2x80x9cP- and P-picturesxe2x80x9d. A P-picture (P frame) is coded by the interframe prediction by referring to an M-preceding picture.
One of the reasons why the B-picture is used, is in general, to improve the prediction efficiency by the bidirectional predictive coding. When the temporal redundancy is reduced by the interframe prediction, the quantity of codes in the case of using the same quantization step is reduced and, moreover, the picture quality in the case of the same compression ratio (coding rate) is improved.
Another reason of using the B-pictures is to reduce accumulation of errors caused by continuous forward predictive coding. When frames to be subjected to forward predictive coding continue, a predictive picture itself which is referred to is an interframe predictive coded frame. Assuming now that, when the reference picture in the predictive coding is an I-picture (I frame), the generation of a coded picture is 1 and the generation of a picture which is coded by referring to a picture of the generation (n) is (n+1), increase in the number of generations with elapsing of time in the case of only the forward prediction is faster than that in the case of using the bidirectional prediction. As a result, quantization errors caused by coding are accumulated and become conspicuous and it causes deterioration in picture quality. By using the bidirectional prediction, this problem is lessened.
Although the bidirectional prediction has such an advantage, increase in the M value (distance between each xe2x80x9cI- and P-picturesxe2x80x9d and xe2x80x9cP-P picturesxe2x80x9d) is disadvantageous for the interframe predictive coding. That is, generally, as the interval between the reference frame used for the interframe prediction and a frame to be interframe predictive coded increases, the correlation between the frames is lowered and the interframe predictability deteriorates.
With respect to the interval between frames of the I- or P-pictures indicated by the M value, the optimum point according to an input picture exists. Consequently, in order to improve the picture quality at the same compression ratio (coding bit rate), it is necessary to optimize the M value and perform coding with the optimized M value.
One of methods for solving such problems is a method of coding a picture while it is dynamically changing the M value. FIG. 10 shows an example of the case of decreasing the M value. Reference numerals in the diagram denote picture types and picture numbers in the input order. FIG. 10 shows a case where coding is performed with the M value of 3 in the beginning, and the M value is switched to 2 and then to 1.
Pictures are coded while the coder is changing the M value to the optimum M value. As shown in the diagram, by changing the M value after the I- or P-picture (P frame) is inputted in the input order or before the I- or P-picture (P frame) is coded in the coding order, pictures can be continuously coded.
FIG. 11 shows the case of increasing the M value from 1 to 2 and then from 2 to 3. In the case where pictures are coded with a large M value, it is necessary to rearrange the order of coding B-pictures. Consequently, there is a frame delay between input of a picture signal and coding. Even if coding is started with a small M value, therefore, it is necessary to start the coding with a frame delay large enough to allow a delay which occurs in the case where coding is performed with the maximum M value which can be dealt by the coding apparatus.
Further, FIG. 12 shows an example of the case where the M value is not continuously changed but is changed from 3 to 1 or from 1 to 3. As shown in the drawing, the M value can be also changed to a value which is not continuous.
One of conventional methods of coding pictures while dynamically changing the M value is disclosed in JP9-294266 A (1997) (hereinbelow, called xe2x80x9cfirst related artxe2x80x9d). This publication discloses an example of a system of changing a distance between frames of intraframe prediction on the basis of a motion vector search range. According to the technique disclosed in the publication, the distance between frames is increased according to the ratio of distribution of motion vectors and the distance between frames is shortened according to a differential value between frames.
For example, as shown in FIG. 13, a case of using a motion vector search range from X13 to Xu3 and coding a current picture with the M value of 2 will be examined. At this time, in the case where a number of motion vectors detected are distributed within the range from X12 to Xu2, which is ⅔ of the search range, it is regarded that the motion vectors can be sufficiently searched in the same search range even when the M value is changed to 3. Consequently, the M value is increased to 3 and a next picture to be coded as a P-picture (P frame) is set as P3. Alternatively, the M value is unchanged and a next picture to be coded as a P-picture (P frame) is set as P2. When the difference between frames exceeds a predetermined value, the M value is contrarily decreased to 1 and a next picture to be coded as a P-picture (P frame) is set as P1.
Another conventional technique is disclosed in JP 10-304374 A (1998) in which the M value is dynamically changed during coding (hereinbelow, called xe2x80x9csecond related artxe2x80x9d). The publication discloses an example of a system of increasing/decreasing the distance between frames for interframe prediction in accordance with the value of prediction efficiency. The prediction efficiency is calculated by using the value of a prediction error and the value of activity.
A disadvantage of the first related art is that the M value cannot be always optimized merely by using the distribution of detected motion vectors since the predictability does not simply depend on only the size of the search range, but there is a case such that a smaller M value is preferable even when motion vectors are within the search range. Another disadvantage is that the M value cannot be optimized only by the value of the difference between frames since the predictability does not simply depend on only the value of the difference between frames but there is a case that the M value may be increased when prediction can be made with a larger M value.
A disadvantage of the second related art is that the M value may not be optimized, even when the value of the prediction error and the value of activity are used. The degree of the percieved prediction error is not considered, since the efficiency of prediction is set as a reference for changing the M value in the art. The degree of deterioration in picture should be measured with respect to a picture obtained as a result of coding an interframe prediction error. The influence of the quantization is not considered in the art, because the activity is measured for the input picture.
It is therefore an object of the invention to optimize the M value by taking a degree of a perceived prediction error into consideration.
The invention is characterized in that, in the case of coding moving pictures to be subjected to interframe motion compensated prediction adapted to intraframe coding, interframe forward predictive coding and interframe bidirectional predictive coding, by calculating predictability of interframe prediction and determining a prediction frame interval of frames to be subjected to forward prediction in accordance with the interframe predictability, moving picture are coded while adaptively changing the prediction frame interval in accordance with the predictability.
According to the present invention, the M value can be properly switched and the quality of a coded picture can be improved since the degree at which deterioration in picture quality by coding is detected is used as a reference.